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1  Executive  Summary 

This  report  summarizes  the  work  done  by  MIT  in  Year  5  (1  February  2000  through  31 
January  2001)  of  the  ONR  Grant  N00014-96-1-0937  entitled  “Training  Spatial  Knowledge 
Acquisition  Using  Virtual  Environments”.  It  has  been  prepared  by  Nathaniel  Durlach  (PI), 
Dr  Thomas  E.  v.  Wiegand,  Andrew  Brooks,  Lorraine  Delhorne  and  Jonathan  Pfautz. 

Much  of  our  effort  during  this  period  has  been  directed  towards  further  refinement  and 
development  of  the  generic  spatial-training  simulation  incorporating  the  World  In  Miniature 
(WIM),  and  the  construction  of  a  photorealistic  model  of  a  building  at  MIT  for  use  in 
experiments  involving  this  simulation.  As  part  of  this  development,  the  entire  simulation 
was  rewritten  in  order  to  achieve  compatibility  with  simulations  in  development  at  other 
ONR-funded  institutions.  Several  improvements  were  made  to  deal  with  fidelity  and  operator 
interface  considerations. 

The  second  major  focus  of  our  attention  has  been  a  program  of  basic  research  into  mental 
factors  that  may  affect  the  acquisition  of  spatial  knowledge.  Experiments  were  performed  in¬ 
vestigating  the  relationship  between  imagined  translations  and  imagined  rotations,  following 
results  reported  elsewhere  in  the  literature.  These  results  have  implications  for  the  structure 
of  coordinate  space  in  egocentric  mental  representations. 

A  white  paper,  “Virtual  Environments  and  the  Enhancement  of  Spatial  Behaviour” 
(Durlach  et  al.,  2000)  was  published  in  the  journal  Presence  during  this  reporting  period. 
This  paper  outlines  a  general  research  and  development  program  in  the  area  of  VE-assisted 
spatial  training  in  collaboration  with  other  researchers  outside  MIT. 

A  paper  detailing  the  Automated  VE  Generation  System  created  in  previous  years  of 
this  grant  was  accepted  for  publication  in  Presence  and  is  currently  in  press. 

In  addition,  with  the  assistance  of  undergraduate  research  assistants,  in  the  last  year  we 
began  investigating  three-dimensional  textural  enhancements  for  the  Automated  VE  Gen¬ 
eration  System,  as  well  as  model  optimizations  for  enhanced  computational  performance  on 
automatically  generated  virtual  models. 

The  body  of  this  report  contains  more  detailed  descriptions  of  the  improved  spatial- 
training  simulation  and  an  introduction  to  the  mental  transformation  research. 
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2  Spatial- Training  Simulation 

In  order  to  investigate  factors  underlying  the  transfer  of  training  of  configurational  knowl¬ 
edge  to  the  real  world,  we  have  been  developing  a  spatial-training  simulation  to  facilitate 
testing  and  evaluation  of  supernormal  VE  training  aids.  The  system  places  trainees  inside  a 
photorealistic  model  of  a  real-world  space,  such  that  they  may  be  trained  by  free  exploration, 
guided  flythrough  or  some  form  of  specific  task,  assisted  by  virtual  training  aids.  At  present 
the  primary  training  aid  is  a  device  known  as  a  World  In  Miniature,  or  WIM.  The  WIM  is 
a  complete  3D  miniature  representation  of  the  immersive  space,  which  appears  to  be  held 
in  the  trainee’s  hand  in  the  VE.  This  is  accomplished  by  means  of  a  6  degree-of-freedom 
motion  tracker  that  the  trainee  holds  in  his  or  her  hand  in  the  real  world.  Trainees  are  able 
to  view  their  own  position  in  the  environment  by  referencing  their  miniature  avatar  in  the 
WIM,  and  can  remove  occluding  detail  from  the  WIM  with  the  use  of  a  self-guided  planar 
cutter  held  in  the  other  hand,  known  as  the  “saw” . 

The  preliminary  WIM  simulation  that  we  developed  in  Year  4  was  implemented  in  a  col¬ 
laborative  shared  3D  environment  named  DIVE.  However,  partly  due  to  this  environment’s 
reliance  on  interpreted  scripting  languages,  and  partly  due  to  the  different  optimizations 
chosen  in  the  automatically  generated  3D  models  from  our  Automated  VE  Generation  Sys¬ 
tem,  we  encountered  severe  performance  difficulties  when  attempting  to  use  our  photore¬ 
alistic  models.  These  difficulties,  in  conjunction  with  our  desire  to  standardise  equipment 
and  software  with  other  ONR-funded  research  groups,  led  us  to  abandon  the  collaborative 
environment  of  DIVE  in  favour  of  a  high-performance  commercial  single-user  simulation 
environment  named  Vega,  based  on  a  port  of  Silicon  Graphics’  Performer.  A  significant 
part  of  the  early  effort  in  this  reporting  period  involved  rewriting  the  entire  spatial-training 
simulation  in  Vega.  This  resulted  in  greatly  increased  rendering  performance  and  enhanced 
programming  capabilities,  as  well  as  enhancing  our  ability  to  share  our  work  with  other 
groups. 

During  and  after  the  recoding  effort,  a  number  of  enhancements  were  made  to  the  WIM 
simulation.  For  example,  the  saw  was  modified  in  several  ways.  In  the  initial  WIM  simula¬ 
tion,  the  saw  would  remove  any  complete  polygon  with  which  it  came  into  contact.  However, 
this  proved  confusing,  particularly  in  the  case  of  large  polygons  such  as  ceilings  and  floors, 
as  users  would  find  the  saw  removing  model  components  quite  spatially  distant  from  the 
plane  of  the  saw.  The  saw  was  thus  upgraded  to  operate  at  the  pixel  rather  than  the  poly¬ 
gon  level.  In  addition,  allowing  the  saw  to  rotate  through  three  unconstrained  degrees  of 
freedom  proved  challenging  for  novice  users,  so  a  saw  access  mode  was  added  to  lock  the  saw 
angles  to  multiples  of  45  degrees,  thus  making  it  easier  for  trainees  to  generate  “standard” 
cuts.  Similarly,  once  a  desired  cut  is  achieved  it  is  difficult  to  move  both  WIM  and  saw  in 
concert  to  preserve  the  model  view  while  adjusting  the  viewpoint,  so  an  option  to  lock  the 
saw  to  the  WIM  was  also  designed. 

In  addition  to  the  simulation  engine,  a  critical  component  of  the  spatial-training  system 
is  a  realistic  virtual  model  of  a  real  world  space  that  is  also  available  in  which  to  train  spatial 
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Figure  1:  Example  view  inside  photorealistic  building  model 


knowledge,  in  order  to  evaluate  training  transfer.  We  therefore  commenced  construction  of  a 
photorealistic  model  of  the  basement,  mezzanine  and  first  floor  of  the  building  in  which  our 
laboratory  resides.  An  example  screenshot,  showing  the  photographic  detail,  is  presented  in 
Figure  1.  We  anticipate  that  the  level  of  detail  present  in  this  model  will  be  sufficient  to 
instil  a  sense  of  presence  in  trainees,  as  well  as  provide  them  with  accurate  cues  to  underpin 
the  transfer  of  training  to  the  real  world  for  testing  purposes. 

3  Mental  Transformation  Research 

We  have  initiated  a  program  of  basic  research  concerned  with  characterizing  and  modeling 
human  performance  in  the  exercise  of  basic  spatial  skills  and  abilities  (BSSAs).  Of  partic¬ 
ular  interest  is  the  difficulty  many  individuals  have  imagining  the  effects  of  translation  and 
rotation. 

In  this  study,  blindfolded  subjects  pointed  to  objects  after  imagined  movements.  The 
imagined  movements  consisted  of  translational  and  rotational  components.  That  is,  the 
subjects  imagined  (they  did  not  actually  move)  that  they  moved  away  from  their  original 
location  (translation)  or  that  they  changed  the  direction  (rotation)  they  were  facing.  Three 
objects  (a  cube,  a  cone,  and  a  ball)  were  placed  in  a  room  at  three  different  angles  from 
the  subject  according  to  one  of  two  floor  plans,  A  or  B.  The  subject  stood  behind  a  cir¬ 
cular  pointer  that  was  mounted  on  top  of  a  dial  marked  with  degree  gradations  along  the 
perimeter.  He  or  she  was  instructed  to  survey  the  floor  plan  and  placement  of  objects  and 
was  then  blindfolded.  The  experimenter  instructed  the  subjects  with  the  imagined  move¬ 
ment  instructions.  For  each  of  six  imaginary  positions  under  each  condition  (translation  or 
rotation),  the  subjects  were  instructed  to  point  to  the  location  of  one  of  three  objects  (i.e., 
the  target  for  the  particular  trial).  The  experimental  set-up  is  shown  in  Figure  2,  where  the 
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top  panels  represent  the  floor  plans  for  the  translation  portion  of  the  experiment  and  the 
lower  panels  represent  the  floor  plans  for  the  rotation  portion  of  the  experiment.  Within 
each  floor  plan  the  3  basic  shapes  represent  the  3  objects/targets,  the  marked  circles  mark 
the  position  where  the  subject  stood  during  the  experiment,  and  the  circles  with  numbers 
within  (translation)  or  beside  (rotation)  them  represent  the  imagined  positions  of  the  subject 
following  the  instructions.  Both  response  time  and  accuracy  of  response  were  measured  for 
all  tests.  Thirty-one  subjects  have  been  tested  and  we  are  in  the  process  of  analyzing  the 
data  and  writing  a  report. 

In  a  second  related  experiment  translation  and  rotation  effects  were  examined  using  a 
simulated  room  design  rather  than  placement  of  the  subject  in  an  actual  room.  In  the 
simulated  design,  the  subject  was  presented  with  a  34"  x  22"  piece  of  cardboard  with  a 
floor  plan  for  the  placement  of  each  of  the  three  objects  (the  floor  plans  were  the  same  as 
in  the  real  room  experiment).  In  this  room  simulation,  a  small  doll  figure  represented  the 
subject’s  position.  The  circular  pointer  used  for  responses  was  mounted  on  top  of  the  doll’s 
head.  From  this  point  on  the  experiment  proceeded  as  described  above,  using  10  subjects 
who  were  a  subset  of  the  original  thirty-one.  While  the  data  analysis  is  not  yet  complete, 
a  preliminary  summary  suggests  that  the  subjects  experienced  difficulty  in  both  translation 
and  rotation  under  the  real  room  and  simulated  room  conditions. 
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Floor  Plan  A  Floor  Plan  B 


Figure  2:  Diagram  of  experimental  setup 
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